DOCUMENT RESUME 



ED 423 713 



FL 801 251 



AUTHOR 

TITLE 

INSTITUTION 
SPONS AGENCY 
PUB DATE 
NOTE 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Kahn, Andrea B.; Butler, Frances A.; Weigle, Sara Cushing; 
Sato, Edynn Y. 

Adult ESL Placement Procedures in California: A Summary of 
Survey Results. Adult ESL Assessment Project. 

California Univ. , Los Angeles. Graduate School of Education 
California State Dept, of Education, Sacramento. 

1994-11-00 

6 5p . ; For related documents, see FL 801 250-253. 

Reports - Research (143) 

MF01/PC03 Plus Postage. 

* Adult Education; *English (Second Language) ; Language 
Proficiency; Literacy Education; Scoring; Second Language 
Instruction; *State Standards; State Surveys; *Student 
Placement; *Testing 
♦California; Placement Tests 



ABSTRACT 



This report presents results of a fall 1993 survey of 164 
adult education agencies in California concerning their placement procedures 
for English-as-a-Second-Language (ESL) students. The survey had two purposes. 
First, the information gathered by a questionnaire (appended) was to provide 
the California department of education with a profile of current ESL 
placement practices, including strengths, weaknesses, and perceived needs. In 
addition, the information was to help inform development of prototype items 
for adult ESL placement as part of a state project to identify appropriate 
placement procedures using the state standards for adult ESL proficiency 
levels. The results reported here summarize information on agency 
demographics, development and use of current placement procedures, 
agency- created and commercially-available (list appended) instruments used, 
administration and scoring, placement decisions and accuracy, strengths and 
weaknesses of current procedures, and the perceived match of the procedures 
to state standards. Contains 3 references. The Placement Procedures 
Questionnaire, a list of responding agencies with ESL programs, and a list of 
commercially available instruments currently being used for ESL placement are 
appended. (MSE) (Adjunct ERIC Clearinghouse on Literacy Education) 
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This report documents the scope and complexities of English as a 
Second Language placement procedures across adult education agencies in 
California. The information presented in the report highlights the varying 
needs of agencies throughout the state and underscores the collective need for 
a viable and equitable placement testing plan that enables educators to 
effectively place students into levels articulated in the English-as-a-Second- 
Language Model Standards for Adult Education Programs. 
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Introduction 



In the Fall of 1993, the California Adult ESL Assessment Project 
conducted a survey of current ESL placement procedures in adult education 
agencies in California. To this end, a questionnaire was developed by project 
staff and distributed to agencies 1 throughout the state by the California 
Department of Education. (See Appendix A for the questionnaire.) The survey 
had two purposes. First, the information gleaned from the questionnaire 
would provide the California Department of Education with a profile of current 
ESL placement practices statewide, including strengths and weaknesses of 
current practices as well as perceived needs. Second, the information would 
help inform the development of prototype items for Adult ESL placement. This 
effort is being undertaken as part of the California Adult ESL Assessment 
Project (Butler, Weigle, & Sato, 1993) which is attempting to identify 
appropriate placement procedures for the proficiency levels articulated in the 
English-as-a-Second-Language Model Standards for Adult Education 
Programs 2 (California Department of Education, 1992). 

Approximately 300 questionnaires were sent to adult education agencies 
in California that receive funding from the federal Adult Education Act, 
Section 321. 3 Of those agencies, 181 responded, with 164 indicating that they 
currently have an ESL program. While specific information from individual 
agencies differed considerably, there was a clear commonality in the attempt 
of agencies to align both course content and placement procedures to the Model 
Standards. 

henceforth in this report, adult education agency/agencies in California will be referred to 
as “agency” or “agencies.” 

^Henceforth in this report, the English-as-a-Second-Language Model Standards for Adult 
Education Programs will be referred to as the Model Standards. 

3 Henceforth in this report, funding from Public Law 100-297, the federal Adult Education Act, 
Section 321 will be referred to as 321 funding. The purpose of these funds is to supplement 
basic skills and ESL programs already offered by adult literacy providers. 



The questionnaire was divided into two broad categories, demographic 
information and placement information. The demographic section requested 
information about agency size, current levels taught, and curricular 
alignment with the Model Standards. The placement information section 
consisted of 13 open-ended questions asking agencies to describe their 
placement procedure and to discuss various aspects of the process, including 
administration, scoring, placement decisions, accuracy, and perceived 
strengths and weaknesses of placement procedures. In addition, there were 
two questions regarding agency willingness to share their placement tests or 
procedures with other agencies or have them evaluated by testing specialists. 4 

Because the questions were open-ended, responses varied widely and 
often overlapped with each other, so that, for example, information about 
accuracy of placement might be found in responses to two or more questions. 
For this reason, the discussion of the placement information in this report 
does not follow the exact order of questions on the questionnaire. Instead, the 
discussion is organized in the following way. First, the development or 
selection of placement procedures is discussed. Next, placement instruments 
currently being used by agencies are described. Finally, issues relevant to the 
placement process such as administration, scoring, accuracy, likes and 
dislikes, and match to the Model Standards are addressed. 

It should be noted that, throughout the discussion, the information 
presented represents the views of only those agencies with ESL programs that 
responded to the questionnaire and may not be representative of all agencies 
throughout California. Furthermore, an attempt has been made to explain 
common trends in questionnaire responses. The explanations provided, 
however, are by no means exhaustive. 

4 The majority of respondents indicated that they were willing to share their tests and 
procedures and have them evaluated. 
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Demographic Information 



The 164 agencies referred to in this report represent adult ESL programs 
throughout the state. These agencies vary greatly not only in the number of 
students enrolled, but also in the student population they serve and in agency 
organization. The demographic information presented in the following section 
is intended to provide a better understanding of these differences, particularly 
with regard to agency size, current levels taught, and degree of alignment 
with the Model Standards. 

Respondents 

Agencies with ESL programs that responded to the questionnaire are 
listed in Appendix B. The positions the respondents hold at their respective 
agencies are summarized in Table 1. 



Table 1 

Respondent Positions 



Position 


Number of 
Agencies 


Administrator 


68 


Coordinator 


48 


Resource specialist 


16 


Instructor 


13 


Other stafP 


16 


No response 


3 




164 



a Other staff positions include registrar, library 
technician, project assistant, counselor. 



The questionnaire was most often completed by ESL administrators (68 
agencies) and coordinators (48 agencies). Resource specialists, instructors, 



and other staff members also completed the questionnaire, but less frequently. 
The differences in the positions held by the respondents show the range across 
agencies of the type of staff familiar with and responsible for placement. 



Agency Size 

There are two important indicators of agency size: the number of 

students currently enrolled and the number of students needing placement per 
term. While the number of students currently enrolled in ESL programs is a 
good indication of agency size, the number of students needing placement per 
term more specifically addresses placement issues. 

Number of students enrolled . Not surprisingly, the number of students 
enrolled in adult ESL programs in California varies greatly across agencies. 5 
As indicated in Table 2, the range varies from fewer than 100 to more than 
10,000. One agency, the Los Angeles Unified School District, reported over 
150,000 students enrolled. 



Table 2 

Number of Students Enrolled in EST, Programs 





Number of Students Enrolled in 
ESL Program 


Number of 
Agencies 


Fewer than 100 


29 


100-500 


50 


501 - 1,000 


21 


1,001-2,000 


24 


2,001-10,000 


25 


More than 10,000 


8 


Agency did not indicate 


7 




164 



5 In California the size of ESL programs is usually determined by average daily attendance 
(ADA). Since agencies were not asked on the questionnaire to provide this information, it is 
not possible to categorize agencies by size according to ADA. 



According to the categories in the table, the greatest number of agencies 
currently has between 100 and 500 students enrolled. However, if taken 
together, an equally large number of agencies (57) indicated an enrollment of 
over 1,000 students. It is clear that substantial numbers of ESL students are 
being served by adult education agencies of all sizes in California. 

Number of students needing placement per term . The number of 
students needing placement per term directly affects the amount of time 
required to administer and score placement instruments and, therefore, often 
influences the selection of placement procedures. Table 3 shows the number of 
students currently needing placement per term. 



Table 3 

Number of ESL Students Needing Placement per Term 



Number of Students Needing 
Placement per Term 


Number of 
Agencies 


Fewer than 100 


36 


100-500 


54 


501 - 1,000 


15 


1,001 - 2,000 


17 


2,001 - 10,000 


8 


More than 10,000 


3 


Agency did not indicate 


31 




164 



While 36 agencies need to place fewer than 100 students per term, the 
majority need to place between 100 and 500 students. Forty-three agencies 
indicated the need to place over 500 students per term with three reporting the 
need to place over 10,000. However, 31 agencies did not respond to this 
question, possibly because open-enrollment policies and fluctuating 



populations make it difficult to determine the number of students needing 
placement at any given time. 

Responses in various sections of the questionnaire indicated that 
many agencies have different placement needs throughout a term. Thus, 
procedures appropriate for the beginning of the term, when there tend to be 
more students, may be quite different from the procedures appropriate during 
the remainder of the term. 

Current Levels Taught 

Respondents were asked to indicate the levels currently taught at their 
agencies. Table 4 summarizes these responses, which are categorized in the 
following way: Model Standards Level Descriptions, Other Proficiency Level 
Descriptions, Number of Levels Taught, and Other. 

While some agencies indicated the names of their courses, others only 
indicated the number of levels offered, thus making it difficult to summarize 
this information. What can be said, however, is that at least 27 agencies are 
using the language of the Model Standards in defining their course levels, 
indicating that an initial step in the alignment process has been taken. 
Comments in other sections of the questionnaire indicated that additional 
agencies have begun this process (see discussion under Alignment Issues), 
but the degree to which course levels at these agencies match the Model 
Standards descriptions cannot be determined from the information provided. 

Twenty-eight agencies indicated teaching beginning and/or 
intermediate levels only. Moreover, of those agencies that are using the 
language of the Model Standards, six are not teaching advanced-high 
courses and one is teaching neither advanced-high nor advanced-low courses. 
It is also likely that the 27 agencies that reported teaching from two to five 



Table 4 

Description of Current Levels Taught 



Number of 



Description of Current Levels Taught Agencies 

Model Standards Level Descriptions 27 

All 7 Model Standards levels 17 

All Model Standards levels except advanced high 6 

All Model Standards levels except beginning literacy 3 

All Model Standards levels except advanced low/high 1 

Other Proficiency Level Descriptions 68 

Beginning, intermediate, and advanced 37 

Beginning and/or intermediate only 28 

Literacy only 2 

Intermediate and advanced only 1 

Number of Levels Taught 59 

2-5 levels 27 

6-7 levels 23 

All levels 9 

Other (includes multi-level classes, individual tutoring) 10 

TOTAL 164 



levels have only beginning- and intermediate-level courses. The fact that 
these agencies do not apparently offer higher-level courses could result from 
a difference in terminology (e.g., advanced skills are taught in courses that 
are multilevel or have different titles); or, more likely, these agencies simply 
do not have sufficient enrollment or funding to support advanced-level courses. 

Very few agencies indicated a one-on-one tutoring situation or 
multilevel classes, although from other responses it is clear that these 
situations are common realities, particularly for agencies that focus primarily 
on basic literacy. For these agencies, placement may be of lesser concern since 
it is fairly easy to make adjustments on an individual basis without having 



to disrupt class schedules and rosters; nevertheless, placement information 
is needed by these agencies in order to appropriately target instruction and 
select materials. 

Although responses varied considerably across agencies, the levels 
currently being taught along with the number of students needing placement 
per term help paint a complex but realistic picture of the varying placement 
needs across agencies. 

Alignment Issues 

While some information about agency degree of alignment with the 
Model Standards was gleaned from the level descriptions above, further 
information was obtained from the questions dealing directly with alignment 
issues. Agencies were first asked to indicate the extent to which their ESL 
courses are currently aligned (completely, partially, or not at all) with the 
proficiency levels in the Model Standards. Then, if their courses were not 
already aligned, they were asked to indicate their expected date of alignment. 

Agency responses to the first question are summarized in Table 5, 
which shows the reported degree of alignment by number of agencies. 

Table 5 

Agency Indication of Degree of Alignment with 

Model Standards 

Number of 



Degree of Alignment Agencies 

Completely aligned 34 

Almost completely aligned 11 

Partially aligned 100 

Not at all aligned 16 

Agency did not indicate 3 



Although “almost completely aligned” was not an option, 11 agencies offered 
this answer to emphasize that they are actively involved in the process of 
aligning or to indicate that their courses have been renamed, but curricular 
changes have not yet been implemented. Complete or almost complete 
alignment was reported by 45 agencies, while the majority (100 agencies) 
indicated partial alignment. Only a small number (16) indicated that they are 
currently not at all aligned. 

Agencies reporting complete or almost complete alignment were 
usually adult schools or community colleges, while those that reported not 
being aligned at all were usually smaller (fewer than 130 students enrolled), 
community-based organizations. In fact, when these agencies were asked 
when they anticipated being aligned with the Model Standards, 10 out of 16 did 
not provide an expected date for alignment. Of these, two agencies did not 
respond at all, but eight provided an alternative explanation. While some 
pointed out that they were not sure what the Model Standards were, others 
were concerned with the difficulty of aligning due to the small number of 
students enrolled or the nature of their program. As one agency explained, 
the “program is so small and fluctuates with the numbers, it is difficult to 
estimate.” Another agency, which works primarily with basic literacy skills, 
clearly stated it has “no plans for alignment.” Whether such problems are due 
to lack of information or different program goals is unclear. Nevertheless, if 
alignment with the Model Standards is a California Department of Education 
goal, efforts should continue to be made to ensure that agencies are aware of 
and have access to the Model Standards. 

Though these issues seem to be most common for agencies that are 
reportedly not at all aligned, a few of the agencies that reported partial 
alignment seem to have similar problems regarding implementation of the 



4 



Model Standards, particularly agencies that have literacy and vocational 
programs. One of these agencies stated that “as a vocational training school, 
alignment depends on the extent that we. can make our vocational curriculum 
meet the Model Standards.” 

Despite the difficulties some agencies may be facing in aligning 
proficiency levels and curricula to the Model Standards, most agencies that 
responded to the questionnaire plan to be aligned well before the July 1995 
deadline required by the state. Expected alignment dates are reported in 
Table 6. 



Table 6 

Expected Date of Alignment for Agencies That Responded to the Questionnaire 



Expected Alignment 


Number of 
Agencies 


Expected Alignment 


Number of 
Agencies 


Already aligned 


34 


Spring 1995 


i 


Soon 


2 


Summer 1995 


7 


In progress 


6 


Fall 1995 


8 


Spring 1994 


16 


Summer 1996 


1 


Summer 1994 


23 


1997 


1 


Fall 1994 


28 


Alternative explanation 


13 


Winter 1995 


5 


No response 


19 








164 



Of the 130 agencies that did not report complete alignment, 47 plan on 
completing the alignment process by Summer 1994 and 41 by Summer 1995. 4 

Only 10 agencies indicated that they plan to align after the July 1995 deadline. 

However, given the number of agencies that did not respond to the question (19) 

or provided an alternative explanation (13), it appears there may be some 4 




10 



13 



* 



difficulty in assessing the amount of time and resources needed to complete 
such a task. 

It should be noted that alignment with the Model Standards is an 
ongoing and complex process which involves a great deal of time and effort. 
Therefore, the expected alignment dates reported most likely reflect initial 
alignment efforts with the understanding that the process will continue as 
agencies fine-tune both course content and placement. 



Development and Use of Current Placement Procedures 

In addition to demographic information concerning agency size, levels 
taught and alignment issues, the questionnaire was designed to help provide 
insight into current placement procedures across agencies. This section of the 
report refers to Questions 2 and 3, which focus on the development and use of 
current placement procedures. Of particular interest is how agencies 
developed or selected their current procedure and how long the procedure has 
been in use. 

Procedure Development and Selection 

Agencies were asked to explain how their placement procedures were 
originally developed or selected. Although there was a wide range of 
responses to this question, some common trends could be detected. One of the 
most common procedures for selecting placement instruments was that of 
group selection or development. Forty-four agencies indicated their placement 
procedure had been developed or selected by a committee of agency teachers 
and/or administrators. Another predominant theme was the role of federal 
and state programs in guiding agencies. Thirty-eight agencies indicated that 
their current procedure had been required or recommended by 321 funding, 



the Greater Avenues to Independence (GAIN) program, the Immigration 
Reform and Control Act (IRCA) program, or most recently the new Model 
Standards. 

A smaller number of agencies (29) indicated that their current 
placement instrument was recommended by Comprehensive Adult Student 
Assessment System (CASAS) staff, while 17 agencies reported that their 
current placement procedure was either recommended or developed by their 
district, a local community college, or another agency. 

In addition to the processes described above, agencies also attended 
workshops, consulted with other agencies, brought in a testing specialist, or 
reviewed available commercial tests. Moreover, the agencies that developed 
their own instruments often reported having reviewed, revised, and field tested 
their instrument before it was adopted. 

Although agencies have arrived at their current placement procedures 
in different ways, it is clear from the questionnaire responses that much 
thought and effort have been put into the development and selection of 
instruments and procedures currently used for ESL placement at adult 
agencies in California. 

How Long Procedure Has Been in Use 

The number of years current placement procedures have been in use 
varies among agencies, possibly due to the time and effort involved in 
developing placement procedures and the resources available to agencies for 
this undertaking. Table 7 summarizes the number of years current placement 
procedures have been in use. 



Table 7 

Number of Years Current Placement Procedure Has 



Been in Use 


Number of Years Procedure 
Has Been Used 


Number of 
Agencies 


Less than 1 


17 


1-3 


42 


4-6 


42 


7-10 


36 


11-15 


9 


More than 20 


3 


Other 3 


10 


No response 


5 




164 



a Other responses include several years, since 
beginning of program, since CASAS was introduced. 



Seventeen agencies have implemented their procedures within the past 
year. The changes at these agencies usually reflect attempts to place students 
into new Model Standards levels more accurately. About half of the agencies 
surveyed have been using their placement procedure for one to six years, with 
the remaining agencies using their procedure for seven years or more. 

Agencies that reported using the same instrument for a number of 
years often indicated that revisions had been made over time. Although a few 
agencies indicated reluctance to change or modify current procedures, many 
commented that placement procedures would need to be revised in order to 
reflect the curricular changes described in the Model Standards. In fact, 
agencies frequently mentioned that they looked forward to receiving guidance 
from the state in this endeavor. 



Placement Instruments 



One of the main goals of the questionnaire was to survey the range of 
placement instruments currently being used to place adults into ESL classes. 
Question 1 asked respondents to indicate the names of instruments currently 
being used or to describe their current placement procedures. The 
instruments described fit into two broad categories, those that are 
commercially available and those that are agency-made. Commercially 
available instruments are tests that have been developed for a general ESL 
population and for which testing materials and scoring information are 
provided by the test producer. Agency-made instruments are tests and other 
forms of assessment that have been developed specifically for the agency by 
agency staff, sometimes with assistance from an outside expert. These 
instruments are designed for a local audience and are intended to meet the 
agency’s specific placement needs. Table 8 shows the number of agencies 
currently using commercially available instruments, agency-made 
instruments, or a combination of the two instrument types. 



Table 8 

Type of Placement Instrument Currently Used 



Type of Instrument 


Number of 
Agencies 


Commercially available only 


57 


Agency-made only 


51 


Commercially available 




and agency-made 


52 


No response 


4 



Fifty-seven agencies reported using one or more commercially available 
instruments, 51 agencies reported using agency-made instruments, and 52 
reported using a combination of the two to place students into ESL classes. 
What follows is a description of the instruments that fall under these two broad 
categories. 

Commercially Available Instruments 

Most agencies (109) are currently using one or more commercially 
available instruments as at least part of their procedure. Those instruments 
used by agencies that responded to the questionnaire are listed in Appendix C. 
They vary widely in form and content and are categorized in the following way: 
tests listed in the Model Standards for group or individual placement (pp. 57- 
58), tests produced by CASAS, 6 tests associated with a textbook series or 
teaching method, and other commercially available tests. 

Tests listed in the Model Standards for group or individual placement . 
In the Model Standards, general testing standards and a list of commercially 
available tests for group or individual placement are presented. Of the 164 
agencies that responded to the questionnaire, 13 are currently using one of the 
commercially available tests on this list. Table 9 shows the tests from this list 
that are currently being used by agencies that responded to the questionnaire. 

The Basic English Skills Test (BEST), which has an oral interview 
section and a literacy skills section, is the most frequently used instrument in 
this group. Five agencies reported using this instrument, though only one 
agency administers both sections of the test: three administer only the oral 
interview and one administers only the literacy section. The other tests in this 



6 Although three CASAS tests are listed in the Model Standards, they are listed as “Instruments 
for Assessing Students’ Achievements” and are not included in the lists of tests for group or 
individual placement (pp. 57-58). 



Table 9 

Tests Listed in the Model Standards for Group or Individual Placement Currently 
Being Used for ESL Placement 



Number of Agencies 



Instrument Reporting Use 

Basic English Skills Test (BEST): Oral and Literacy 1 

Basic English Skills Test (BEST): Oral only 3 

Basic English Skills Test (BEST): Literacy only 1 

Combined English Language Skills Assessment (CELSA) 2 

Comprehensive English Language Tests (CELT) 1 

Michigan Test a 1 

Structure Tests - English Language (STEL) 2 

Test of English Proficiency Level (TEPL) 2 

13 



a More specific information regarding which form of the Michigan test is being 
used at this agency could not be obtained in follow-up interviews. 



category are administered at two agencies with the exception of the 
Comprehensive English Language Test (CELT), which is currently being used 
at one agency. 

The Test of English Proficiency Level (TEPL), which has an oral and a 
written component, is the only test in this category which is currently being 
used as the sole placement instrument at one of the agencies. The remaining 
tests are being used in conjunction with other instruments either to complete a 
battery of tests or to address specific placement needs for upper- or lower-level 
students. For example, the BEST oral interview provides an indication of basic 
aural/oral ability and is often used to supplement reading tests. Because the 
BEST focuses on basic language skills, it is being used primarily at the lower 
levels. In contrast, the CELT and the Combined English Language Skills 



Assessment (CELSA), which focus primarily on structure, are often used to 
complete student profiles at the advanced levels. 

CASAS instruments . Of the commercially available instruments being 
used to place ESL students, CASAS tests are by far the most frequently 
administered by agencies that responded to the questionnaire. Table 10 lists 
the CASAS instruments currently being used and indicates the number of 
agencies reporting their use. 



Table 10 

CASAS Instruments Currently Being Used for ESL Placement 



Instrument 


Number of Agencies 
Reporting Use 


Appraisals 

Adult Life Skills Appraisal 


6 


ESL Appraisal 


21 


IRCA Appraisal 


9 


Customized CASAS tests 


7 


Life Skills Survey Achievement Tests 


11 


Other CASAS tests 


26 




a 

© 

00 



a Five agencies are currently using two different CASAS tests. 



Of the 164 agencies surveyed, 75 indicated using one or more CASAS 
tests to place their students (five agencies are using two different CASAS 
tests). The CASAS Appraisal forms-ESL Appraisal, Adult Life Skills 
Appraisal and IRCA Pre-enrollment Appraisal— are currently being used by 36 
agencies. These tests are designed for placement, but the intended audience 
and purpose of these tests differ. 

Twenty-one agencies reported using the ESL Appraisal, which is 
intended for use with non-native speakers of English and assesses primarily 



listening and reading comprehension. 7 Six agencies reported using the Adult 
Life Skills Appraisal, which tests reading and math and is intended for use 
with Adult Basic Education (ABE) students, although CASAS guidelines 
indicate that the Adult Life Skills Appraisal can also be used for placement 
with ESL students (CASAS Life Skills Listening Test Administration Packet, 
February 1993, p. 4). 8 

The IRCA Appraisal assesses listening and reading comprehension 9 
and is also designed for non-native speakers. The intended use of this test is to 
provide an initial assessment of English language ability in the context of 
United States history and government (CASAS IRCA Pre-enrollment 
Appraisal Manual, p. 1 ). At one time, California agencies that were 
participating in the Amnesty Program were required to administer this test in 
order to qualify for federal funding. Questionnaire results indicate that nine 
agencies are still using this instrument even though the Amnesty Program no 
longer exists and curricular goals have changed. 

Customized CASAS tests are constructed by CASAS specialists in 
collaboration with staff from the agency requesting the test. Through this 
process agencies can construct a placement test that fits the needs of their 
student population by selecting items from the CASAS item bank from a 
specific content area and at a specified level of difficulty. Of those agencies that 
responded to the questionnaire, seven are currently using a customized 
CASAS test, and in each case the test is designed to assess reading 
comprehension. 

7 Speaking and writing components are also part of the ESL Appraisal, but questionnaire 
results indicate that they are rarely used. 

8 An overview of the CASAS Appraisal tests, including the information about the intended 
audience for these tests, was found only in the Life Skills Listening Administration Packet, 
not in the administration information for the ALS or ESL Appraisal. 

9 The IRCA Appraisal also assesses speaking and writing, but as with the ESL Appraisal, these 
components are rarely administered. 



CASAS Survey Achievement Tests are level tests designed to assess 
either listening or reading comprehension; each level has two forms for use in 
pre- and post-instructional testing. CASAS Survey Achievement Tests are 
currently being used for placement purposes by 11 agencies that completed the 
questionnaire even though these tests are designed to evaluate student 
progress and are not recommended as placement instruments (CASAS Life 
Skills Listening Test Administration Packet, February 1993, pp. 3-4). 

The final group of CASAS instruments includes other CASAS tests such 
as those developed for GAIN and the CASAS Locator Test. There are 26 
agencies that are currently using one of these CASAS instruments, which 
were categorized under this heading either because they are infrequently used 
or, more commonly, because it was not clear exactly which CASAS 
instrument is being administered. Follow-up interviews were conducted with 
a number of agencies in an attempt to more clearly specify the CASAS 
instrument being used. Some clarification was obtained, though in most cases 
the exact instrument remained unclear. 

Tests associated with a textbook series or teaching method . Tests 
associated with published textbook series or teaching methods are also being 
used to place ESL students into adult education classes. Some series, such as 
Side by Side and Real Life English, have developed placement tests to 
determine the appropriate level or book for students to begin their studies. In 
the same way, the Literacy Volunteers of America, a literacy organization, 
has developed a placement instrument that determines where students should 
begin their training. These instruments, shown in Table 11, are currently 
being used at 15 agencies, with one agency using two of the instruments to 
place its students. 



Table 11 

Tests Associated with a Textbook Series or Teaching Mpthnd 



Instrument 


Number of Agencies 
Reporting Use 


English for a Changing World Placement Tests 


i 


ESL Adult Literacy Scale 


i 


Intercom 2000 Grammar Test 


i 


Laubach Way to Reading 


4 


Literacy Volunteers of America Placement 


2 


New Horizons Placement Test 


1 


Pathways to English Placement Tests 


1 


Practical English Tests 


1 


Real Life English Competency Test 


1 


Side by Side Placement Test 


3 

16 a 



a One agency is currently using two of these tests. 



These instruments tend to be used by smaller, community-based 
programs. Often such programs have small classes or one-to-one tutoring 
situations, and the coordinators may find it easier to decide which text in the 
teaching series students should begin with based on the results of the 
associated placement test. 

Other commercial tests . The last category, other commercial tests, 
consists of 16 tests that cannot be categorized in the above groups. 10 The use of 
these tests is not very widespread: Only 16 agencies reported using one or 
more of them. Two tests-Quick Assessment, a speaking test, and the Idea 
Oral Language Proficiency Test, a reading structure test-are used by two 
agencies each; the rest of these tests are used by one agency only. In two cases, 



10 For a list of the other commercial tests currently being used by agencies that responded to the 
questionnaire, see Appendix C. 



two tests in this category are used by the same agency: One agency uses both 
the ESL Oral Assessment and the ESL Structure Test, while another agency 
uses the Alemany Grammar Test and the Sullivan Reading Test. 

Agencv-made Instruments 

In addition to a wide range of commercially available instruments, 
several different agency-made instruments are being used to place students 
in ESL classes. These tests vary greatly in test format and skill areas assessed 
and, as mentioned above, are often administered in conjunction with one of 
the commercial tests already discussed. In fact, of the 103 agencies currently 
using agency-made tests, 52 use them to supplement commercial 
instruments, while 51 agencies rely on them exclusively to place students. 
Agency-made instruments can be divided into two broad categories: aural/oral 
procedures and written procedures. 

A ural/oral procedures . Respondents reported using a number of 
different oral procedures, such as interviews, oral reading tests, dictations, 
oral literacy screens, and first language (Ll) interviews. These procedures are 
primarily intended to assess speaking ability and listening comprehension, 
though some instruments assess literacy skills as well. Table 12 lists the oral 
procedures currently used and shows the number of agencies at which they 
are used. 

Oral interviews are by far the most widely used agency-made procedure, 
with 85 agencies indicating their use. Interviews vary greatly across agencies 
in length, format, and, above all, intended use. Given the large number of 
interview types reported, it would be impossible to discuss all the different 
variations in this report. Therefore, two representative interview types are 
summarized. 



Table 12 

Agency-made Instruments: Aural/Oral Procedures 



Aural/Oral Procedure 


Number of 
Agencies 


Oral interview 


85 


Oral reading test 


3 


LI interview/assessment 


2 


Dictation 


1 


Oral literacy screen 


1 




92 



Some interviews are intended to be quick screening devices where the 
student’s ability to answer a few personal questions is sufficient to determine 
whether further testing is needed or if the student should be enrolled in a 
literacy class. This type of interview is generally assessed holistically by the 
test administrator, and criteria for making placement decisions may or may 
not be formalized. Decisions depend on the administrator’s past experience 
and familiarity with the levels being taught. 

Other agencies have developed more formal procedures where students 
respond to personal questions in addition to questions referring to a picture 
prompt. Often these agencies have developed rating scales or other criteria by 
which the tester can make placement decisions. Though this type of oral 
interview is sometimes used as a screening device to determine whether 
further testing is needed or to determine which specific test to a dmini ster, it is 
usually used to provide additional information in completing student profiles. 

These two types of interview procedures described by the respondents 
represent the extremes (from least structured to most structured). Other 
variations in current interview procedures mentioned above fall somewhere on 
the continuum between these two interview types. 

31 



Written procedures . The written procedures described by the respondents vary 
in length, format, and intended use across agencies. Procedures in this 
category include elicited samples of student writing as well as tests containing 
some combination of the following item types: multiple-choice, form 

completion, cloze, short answer, and fill-in-the-blank. These written 
procedures are intended to assess reading comprehension, knowledge of 
English vocabulary, writing ability, or any combination thereof. From the 
responses provided, the format and skills tested in a given procedure were not 
always clear; in these instances, the tests were categorized under the more 
general heading, written tests. Table 13 summarizes the different written 
procedures currently in use and shows the number of agencies which use 
them. 



Table 13 

A^encv-made Instruments: Written Procedures 



Written Procedure 


Number of 
Agencies 


Multiple-choice 


15 


Writing sample 


13 


Form completion 


12 


Written test 


11 


Cloze 


6 


Short answer 


4 


Fill-in-the-blank 


4 


Job survey 


1 


Level tests 


1 

67 



Of the agencies that administer written agency-made tests, 13 currently 
elicit writing samples as part of their placement procedures. The writing 



samples are always used in conjunction with other assessments in order to 
obtain a better “overall picture of student abilities,” particularly at the upper 
levels. One agency explained that the writing sample was implemented upon 
request of the upper-level teachers, while other agencies have only those 
students who place at an intermediate or higher level produce a writing 
sample. These comments reflect the need some agencies have for additional 
information about higher level students in order to place them accurately. 

Alternative procedures . Alternative procedures are also being used to 
place students. Two agencies reported a placement procedure that takes place 
over a period of time ranging from a few hours to one week. Students are 
initially placed in an orientation class where their language abilities are 
assessed. At the end of the orientation class, students are placed with a high 
degree of accuracy into the level appropriate to their language abilities. These 
agencies seem to be enthusiastic about this process because it allows them 
more time to assess student language proficiency and needs. In contrast, two 
agencies do not assess students for placement at all. These agencies allow 
students to self-select the course they wish to attend in hopes that doing so will 
promote greater student motivation and attendance. 



Administration and Scoring 

When deciding on the feasibility of any placement procedure, several 
factors must be taken into consideration, including the administration and 
scoring of the instruments. This section of the report summarizes current 
practice with regard to administration and scoring of placement instruments. 
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Administration 



Agencies were asked to provide information on two issues related to the 
implementation of placement procedures: how often administration takes 
place and how much time is required for the placement process. These issues, 
covered by Questions 4 and 8, are discussed below. 

How often procedure is carried out . Question 8 asked agencies to report 
how often their placement procedure is carried out. The purpose of this 
question was to determine the frequency with which placement tests are 
administered in any given term. Not all agencies interpreted the question in 
the same way. While some agencies interpreted the question as intended, 
others understood it to mean how often the test is administered to each 
student. 

For the agencies that interpreted the question as it was originally 
intended, the majority reported that they have an open-enrollment policy 
which allows students to register at any given time in the term. Most of these 
agencies administer their placement test at the beginning of the term when 
the majority of students enroll and then on an individual basis throughout the 
term whenever a new student enrolls in the program. This often means 
testing takes place on a daily basis. However, some agencies only adm ini ster 
their placement test on designated days. A smaller number of agencies did not 
mention having an open-enrollment policy and thus only indicated testing 
students for placement at the beg innin g of each term. 

Time required to administer placement test . The amount of time 
required to administer placement tests is critical to most agencies given the 
number of students that need to be placed and the personnel available. 
Administration times vary depending on the number of students being tested, 
student ability, and the nature of the test. 



Agencies reported the amount of time required to administer their 
placement test in different ways: Some reported the time required for 

individual administration, others indicated the total time required for group 
administration. In either case, there is clearly an attempt across agencies to 
keep test time to a minimum: up to five minutes per student for individual 
administration and between 15 minutes to an hour for group administration. 
A small number of agencies (33), however, reported needing between one and 
three hours to administer their placement test. One agency reported that a full 
week is needed to complete their placement process. 

Although only actual test administration times are reported, a few 
agencies also indicated the total time a student would need to be present. For 
example, one agency reported that the actual process only takes 20 minutes, 
but students often wait over an hour to be tested. Thus, both actual 
administration time and total wait time for students are factors that need to be 
considered in order to streamline the placement process. 

Scoring 

Another important issue related to placement procedure selection is 
scoring. Given limited time and personnel, scoring needs to be as fast and 
easy as possible and, at the same time, assure accuracy. The following section 
addresses scoring methods and time, Questions 5 and 6. 

Scoring method . In any test situation, the scoring procedure used 
depends on various factors such as test format and the skill tested. Machine 
scoring is only appropriate for items with selected responses, while hand 
scoring is appropriate for both selected and constructed responses. 11 How a 



11 Selected responses require students to select from a number of provided responses. 
Constructed responses require students to generate their own responses. 



procedure is scored directly affects the amount of time required for scoring. 
Table 14 summarizes current scoring practices. 



Table 14 

Scoring Method for Current Procedures 



Scoring Method 


Number of 
Agencies 


Hand 


95 


Machine 


30 


Hand and Machine 


21 


Other a 


12 


No response 


6 




164 



a Other responses include not scored, scored 
holistically, teacher judgment, u nkn own. 



Though machine scoring is usually considered to be faster than hand 
scoring, the majority of the agencies (95) that responded to the questionnaire 
score their placement instrument by hand. Thirty agencies score their 
placement instrument by machine, and 21 agencies use a combination of the 
two methods. Some agencies mentioned other ways of scoring such as holistic 
appraisal or teacher judgment. 

Time required for scoring . Although most placement instruments are 
scored by hand, agencies clearly attempt to minimize the time required to 
score them. Reported scoring times range from 10 to 30 minutes, with only 
eight agencies reporting that the scoring process takes more than one hour. 
However, the scoring time reported may refer to either group or individual 
administration. In the latter case, the total scoring time would increase 
proportionally as the number of tests to be scored increases. Although the 
information gathered from the questionnaire was inconclusive, it is apparent 



that agencies are concerned that the scoring process be as manageable and 
expedient as possible in order to promptly place students in appropriate levels. 



Placement Decisions and Accuracy 

Once instruments are administered and scored, decisions must be made 
about student placement. Question 7 asks agencies to report how these 
decisions are made while Questions 11 and 12 deal with the accuracy of these 
decisions and the frequency with which misplacements and subsequent 
adjustments occur. A discussion of the responses to these three questions 
follows. 

How Placement Decisions Are Made 

Agencies that responded to the questionnaire indicated that placement 
decisions are currently made by using one or more of the methods presented in 
Table 15. 



Table 15 

Placement Methods Used 



Placement Method 


Number of 
Agencies 


Cutoff scores a only 


88 


Cutoff scores with profiles or 
tester judgment** 


38 


Student profiles only 


7 


Drop-off point 


5 


Tester judgment only 


3 


Other 


5 


No response 


18 
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a Fixed score ranges which determine placement. 
^Usually based on an oral interview. 



For the agencies surveyed, placement decisions are made primarily on 
the basis of cutoff scores. They are used exclusively at 88 agencies to place 
students into appropriate levels and in combination with smother method at 38 
agencies. When used in combination, cutoff scores either complete student 
profiles, which are usually based on a battery of tests across skill areas, or 
support tester judgment. 

Cutoff scores are most likely used at a large number of agencies because 
they allow placement decisions to be made quickly and easily. Cutoff scores 
can be arrived at in a variety of ways. Test producers often provide cutoff 
scores for commercially available instruments. However, some agencies opt to 
establish their own cutoff scores either with commercially available 
instruments or with their own agency-made instruments to better match 
program needs. 

Though the majority of agencies that responded to the questionnaire use 
cutoff scores to place students into ESL classes, a few agencies are also using 
alternative methods to make placement decisions. As mentioned above, tester 
judgment or student profiles are used in conjunction with cutoff scores at 38 
agencies and exclusively at three and seven agencies respectively. Tester 
judgment is often used when less-formalized oral interviews are part of the 
placement process. Interview results are used either to confirm placement 
decisions that have been made based on another instrument or to direct 
students to higher or lower levels of testing. Student profiles are usually used 
when a battery of tests that focus on different skill areas is administered. This 
type of profile consists of the individual scores for each of the skill areas tested. 
If student performance across skill areas varies greatly, agencies can decide 
where the emphasis should be placed in terms of making appropriate 
decisions. 



In addition to cutoff scores, tester judgment, and student profiles, five 
agencies make placement decisions based on how much of the test students 
complete (referred to as “drop-off point” in the table). Some placement 
instruments are organized in such a way that test items become progressively 
more difficult. The point in the test where students no longer attempt to 
answer questions or begin getting most items wrong determines their 
placement. This method seems to be common in the oral interviews, and other 
comments throughout the questionnaire indicate that it may be used more 
frequently than reported here. 

Accuracy of Current Placement Procedures 

It is important for both ESL staff and students that placement decisions 
be accurate in order for students to settle in their classes as quickly as possible. 
Students become attached to their classes rather quickly and, once placed, are 
often reluctant to move. Accurate placements eliminate the extra 
administrative work involved in transferring students, and more importantly, 
allow agencies to avoid situations in which students become either frustrated 
or bored by being placed inappropriately. Because funding is based on student 
attendance, accuracy of placement is critical in that it directly impacts student 
retention. Question 11 asked agencies to report on the accuracy of their 
current placement procedures. Their responses are summarized in Table 16. 

Though many agencies seem to be satisfied with the accuracy of their 
procedures (76 agencies described their placement procedures as accurate to 
very accurate), 50 agencies expressed reservations about their placement 
instruments, with 39 reporting their current placement procedures as not 
completely accurate and 11 as not accurate at all. However, it should be noted 
that 13 agencies did not respond to the question and 25 addressed issues related 
to accuracy, but did not specify the degree of accuracy. For example, 11 



agencies reported that the accuracy of the instrument depends on the person 
administering it: “If [the test administrator is] well trained and experienced, 
the process works well.” Tester training is always critical in terms of both 
effective administration and effective scoring. 



Table 16 

Decree of Accuracy of Placement Instruments 
Currently Being Used by Questionnaire Respondents 







Degree of Accuracy 


Number of 
Agencies 



Very accurate 


18 


Accurate 


58 


Not completely accurate 


39 


Inaccurate 


11 


Other 


25 


No response 


13 
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Another issue related to placement accuracy is that students with 
varying abilities are often placed in the same class. This usually happens 
because students do not perform equally well across skill areas. For instance, 
students who are strong in listening and speaking may be weak in reading 
and writing or vice versa. Although there is no way to create a completely 
homogenous class and, in fact, some differences may be beneficial to class 
dynamics, it is clearly a problem when disparities are so great that all needs 
cannot be met. This problem may be minimized by using multiple measures to 
place students as recommended in the Model Standards (pp. 9, 54) so that 
students are not placed on the basis of one skill area only. 

A few agencies also commented on problems associated with placing 
higher- and lower-level students with the same instrument. Specifically, 



agencies mentioned the difficulty in discriminating at the upper levels, that is, 
between advanced low and advanced high, and at the lower levels, between 
beginning low and beginning high. These problems are not surprising since it 
is difficult for a single instrument to discriminate across the entire proficiency 
range described in the Model Standards. An instrument that discriminates at 
the highest levels may be inappropriate for use at the lower levels and vice 
versa. 

Misplacement and Subsequent Adjustment 

Some misplacements inevitably occur with every placement procedure. 
The goal is to minimize the number of students whose level is not accurately 
identified. In Question 12, agencies were asked to first report how often 
misplacements occur and then how subsequent adjustments are made. Table 
17 summarizes the first part of this question. 



Table 17 

Frequency of Misplacements 



Frequency of 
Misplacements 


Number of 
Agencies 


Never 


2 


Seldom (1-2%) 


53 


Occasionally (5-10%) 


23 


Frequently (15-25%) 


12 


Other 


14 


No response 


60 
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Almost half of the agencies (78) indicated that misplacements occur 
either never, seldom, or occasionally, with only 12 agencies reporting that 
misplacements occur frequently. On the surface, these figures seem to 



indicate that misplacement is not a serious problem; however, given that 60 
agencies did not respond to this part of the question, it is difficult to draw 
conclusions with any degree of confidence. 

While a significant number of agencies did not provide information 
regarding the frequency of misplacement, all but 11 agencies reported on the 
way subsequent adjustments are handled. Over half the agencies (94) 
indicated relying on some form of teacher input for making placement 
adjustments. Although usually initiated by teachers, some adjustments are 
also student initiated. In either case, the teacher simply recommends the 
appropriate level and adjustments are made, though a few agencies 
emphasized the fact that adjustment decisions are made by the student and 
teacher together. 

At 22 agencies, students are retested if an error in placement is 
suspected. The new test results are used either to confirm original placement 
decisions or to make appropriate changes. Though some agencies retest with 
the same instrument, others indicated that a different instrument, such as a 
level exit test, is used to reevaluate student proficiency levels. 

Misplacements and subsequent adjustments are less critical at some 
agencies due to the size and nature of the ESL program. In fact, 12 agencies 
reported that making adjustments is not a problem at all due to the tutorial 
nature of the program. In these cases, when misplacements are detected, 
tutors merely make changes in the materials being used. Adjustments are 
also not a problem for seven agencies that have multilevel classes since 
changes are simply made in the internal class groupings. Students thus do 
not have to be transferred from one class to another. 



Strengths and Weaknesses of Current Placement Procedures 



In Questions 9 and 10, agencies were asked to discuss what they like 
about their placement procedures and what problems, if any, they have with 
them. Responses to these questions varied greatly across agencies and 
touched on several key issues related to placement of adult ESL students. 
These issues have been grouped under the following headings: administration 
and scoring, test content, placement and accuracy, and student issues. 

Administration and Scoring 

Issues related to the administration and scoring of placement 
procedures were mentioned by a large number of respondents when 
discussing the strengths and weaknesses of their placement procedures. 

A large number of agencies commented on the amount of time required 
to administer their test and the ease with which it is administered. While 61 
agencies like their procedure because it is quick and easy to administer, 19 
agencies reported that their placement procedure takes too long and is at times 
“frustrating and cumbersome.” This is clearly related to the issue of whether 
or not the procedure is appropriate for large-scale use. In fact, six agencies 
were pleased with their ability to place large numbers of students with their 
procedures, while eight felt that their procedures are inappropriate for large- 
scale placement. 

A number of issues dealing with the personnel required for placement 
testing emerged from questionnaire responses. At ten agencies the large 
number of personnel required to administer their current placement test is 
viewed as a problem. By contrast, at one agency the limited number of 
personnel required to carry out placement procedures is seen as a strength. 
The amount of training required for personnel to adequately administer 
placement tests is another important issue. Though only 16 respondents 



reported that tester training is a problem at their agency, responses 
throughout the questionnaire indicated that many agencies share this view. 
Although agencies recognize the need for proper tester training in order to 
obtain better placement results, they are often unable to rigorously undertake 
this process due to limited resources (i.e., time and money). 

The issue of tester training is further complicated by the fact that some 
test administrators may not be familiar with the ESL program at the agency 
and, at times, are work-study assistants who are frequently non-native 
speakers of English. When placement procedures are carried out by these 
individuals, even more rigorous training is required in order for accurate 
placement decisions to be made, particularly when oral interviews are 
conducted. Moreover, training these assistants is not always cost effective as 
they often stay with the agency for only a short period of time. In such cases, 
agencies may be forced to simplify their procedures so that administrative or 
work-study assistants can handle the placement process. 

Other issues related to test administration and scoring were raised by a 
few agencies. The cost of purchasing placement instruments and funding 
staff to administer and score these instruments is of concern. The quality of 
the testing environment and the need for a testing room are problems for a few 
agencies. Agencies also mentioned the need for test instructions that are easy 
for students to understand as well as test results that are easy for agencies to 
interpret in order to make fast and accurate placement decisions. Finally, 
some agencies mentioned a need for flexible placement procedures that are 
feasible both at the beginning of the term, when there are large numbers of 
students that must be placed, and throughout the term, when smaller 
numbers of students must be placed on an ongoing basis. 



Test Content 



Only a small percentage of total respondents provided feedback 
regarding test content when discussing likes and dislikes of their placement 
procedures, but several content issues were raised. 

The most commonly mentioned content issue was skill area coverage: 
16 agencies reported that they like their test because it covers the skill areas 
that they feel are appropriate for their curriculum, while 23 agencies indicated 
that their test lacks coverage of one or more skill areas that they believe are 
important. Another area of concern is the match between test content and 
course curricula. Five agencies cited the match of their test content to course 
curricula as a strength or mentioned that test results helped inform 
instruction. On the other hand, eight agencies mentioned the lack of 
correspondence with curricula or the Model Standards as a weakness in test 
content. 

A related issue is the extent to which a test addresses life skills. Five 
agencies listed the life-skills orientation of their placement instrument as a 
strength, while one agency reported that their “tests are not performance or 
task oriented and are therefore limited in providing feedback on the students’ 
capabilities in real-life situations.” 

Another issue related to test content is the communicative nature of 
agency placement procedures. Only two agencies were concerned about the 
noncommunicative nature of their test; however, this issue was frequently 
mentioned with regard to the match of their placement procedures to the 
Model Standards (see Match to the Model Standards for further discussion). 

Other comments related to test content are the need for problem-solving 
exercises, lack of variety in test content, and the inclusion of items that 
assume cultural knowledge. The issue of cultural knowledge in a language 
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test is complex because language is always used within some cultural context. 
The challenge is to develop or select language tests that minimize the impact of 
cultural knowledge that is not relevant to the kind of language being assessed. 

Placement and Accuracy 

Agencies often mentioned issues related to student placement and the 
accuracy of student placement when discussing what they like and dislike 
about their current placement procedures. Though 45 agencies reported that 
what they liked- most about their placement procedure is that it is accurate and 
reliable, 18 agencies considered their instruments to be inadequate and, at 
times, unreliable. Moreover, five agencies cited lack of discrimination between 
levels as one of the weaknesses of their placement instrument. These agencies 
noted that their instrument fails to discriminate either at higher or lower 
levels. 

A few agencies pointed out that placement decisions are often subjective. 
This issue is related to tester training and clearly affects the accuracy of 
placement decisions. Other agencies cited the overuse of placement 
instruments as a problem because students who are familiar with the items 
may perform better than they otherwise would. As a result, students may be 
placed at a level that is too high for their true proficiency. A few agencies also 
reported problems with placement tests that have a multiple-choice format 
because they feel students can easily guess the correct answer and be placed at 
too high a level or, conversely, students may be placed at too low a level because 
of a lack of familiarity with the scannable answer sheets often used with this 
format. 

Another problem agencies have with the accuracy of their placement 
decisions is that some students may be reluctant to speak during oral 
interviews or do not answer questions to the best of their ability due to test 



anxiety. On the other hand, some students intentionally perform poorly in oral 
interviews in order to remain at a lower level. This usually happens when 
students want to start at the lowest level to avoid missing anything, or when 
they want to stay with friends or an instructor at a lower level. 

Student Issues 

In discussing what they like most about their placement procedures, 
several agencies brought up issues related to student needs. One important 
issue is that placement instruments need to be as non- threatening as possible. 
Because of the anxiety most students feel in test situations, it is important that 
students be put at ease in order to demonstrate their true ability. Thus, 
directions should be easy to follow, tasks should be manageable, and the 
difficulty level should be appropriate for the population being tested. Fourteen 
agencies reported that what they like most about their current procedure is the 
fact that it is non-threatening to students, and seven other agencies liked 
features that make their procedure non-threatening (e.g., the directions are 
simple and students can stop whenever the questions become too difficult). 

With regard to oral interviews in particular, agencies also like the fact 
that their placement instruments are personal or individualized. Twelve 
agencies indicated that this is what they like most about their procedure. For 
example, one agency feels that with their oral interview “there is an 
opportunity for students to give input and express their goals and needs for 
appropriate placement.” Agencies also like the one-to-one format of most oral 
interviews because students feel as if they are getting more personal attention 
and the support and encouragement they need. 

While approximately 20% of the agencies (36) that responded to the 
questionnaire reported that they have no problems with their current 
placement procedures, for the majority of agencies, issues regarding 



administration and scoring, test content, placement and accuracy and other 
student issues have yet to be resolved. Each of the individual concerns 
discussed above was raised by only a small number of agencies, when taken as 
a whole they present a comprehensive picture of considerations affecting the 
development and selection of procedures for placing ESL students in agencies 
throughout California. 



Match to the Model Standards 

In Question 13, agencies were asked to indicate to what extent, in their 
opinion, their current placement procedures match the content of the Model 
Standards in terms of communicative approach and proficiency levels. Table 
18 summarizes agency responses. 



Table 18 

Perceived Match of Current Placement 
Procedures to the Model Standards 



Degree of Match 
to Model Standards 


Number of 
Agencies 


Matches 


41 


Partially matches 


57 


Doesn't match at all 


22 


Unsure 


8 


In process of revising 


2 


Other 


21 


No response 


13 




164 



While 41 agencies indicated that their placement procedures match the 
Model Standards in both communicative approach and proficiency levels, 57 



agencies reported that their procedures only partially match the Model 
Standards. Of those agencies that reported a partial match with the content of 
the Model Standards, many indicated a match in proficiency level, but not in 
communicative approach. One agency explained that their procedure “seems 
to match well in the area of language proficiency but not in the communicative 
approach because it does not test for that.” This may be because agencies 
consider it difficult to find or develop a placement instrument that is 
communicative in nature. One respondent emphasized this point, stating that 
she has “never seen a test that places well and is communicative.” Another 
explained that the test content can be made to match the Model Standards, “but 
a better way to evaluate communicative skills is needed.” However, some 
agencies indicated that their procedures are in the spirit of the Model 
Standards, though their proficiency levels are not yet aligned. 

A smaller group of agencies (22) reported that their procedures do not 
match the content of the Model Standards at all, usually because they were 
“developed prior to the Model Standards” and have not yet been modified to 
correspond to the Model Standards. As one agency stated, “We are just 
embarking on the task of aligning our courses with the stated proficiency 
levels; our tests will, of course, need to follow that alignment.” Along similar 
lines, two agencies reported that they are currently revising their placement 
procedures to reflect the content of the Model Standards. 

Finally, eight agencies indicated that they are unsure whether their 
procedures match the Model Standards, either because they are unaware of 
the Model Standards or because they feel that they cannot address placement 
issues until they have finished the alignment process. 

The information presented in this section reveals that agencies are very 
much aware of the need for placement procedures that match the Model 



Standards both in terms of proficiency levels and in communicative content. 
However, changes in assessments frequently lag behind curricular changes. 
As agencies continue aligning their course levels to the Model Standards, it 
will be important to consider the degree to which their placement instruments 
are both communicative in nature and accurate in placing students 
appropriately. 



Summary and Conclusions 

As stated at the beginning of this report, the purpose of the survey was 
twofold. The first purpose was to provide the California Department of 
Education with a profile of current adult ESL placement practices and the 
second was to inform the development of prototype items appropriate for ESL 
placement. While it is apparent from this undertaking that agencies 
recognize the role of placement in aligning with the Model Standards, the 
range of responses throughout the questionnaire clearly illustrates the varying 
placement needs of adult ESL agencies statewide and the different ways these 
needs are being addressed. Differences across agencies stem from a variety of 
sources, such as the number of students needing to be placed per term, the 
student population served, and the amount of resources available for the 
development, administration, and scoring of placement-instruments. Though 
some differences may become less pronounced as agencies complete the 
alignment process, many will remain, necessitating the development of 
placement instruments that acco mm odate the full range of agency needs. 

The profile of adult ESL placement practices obtained from the 
questionnaire responses has helped establish parameters in three areas 



which are guiding the prototype development effort 12 : student issues, 

operational constraints, and content considerations. Responses throughout 
the questionnaire emphasized the importance of placement instr um ents that 
are sensitive to student needs. Agencies preferred instruments that elicit 
student language in a non-threatening manner and indicated that issues 
related to level of difficulty, clarity, and test familiarity should also be 
considered. 

Because agencies need to place large numbers of students in a short 
period of time and typically have limited resources to do so, it is imperative that 
efforts be made to streamline the placement process. In practical terms, this 
means keeping the amount of time required to administer and score 
placement tests to a minimum, avoiding instruments that require extensive 
tester and scorer training, and identifying instruments that are appropriate 
for placement both at the beginning of the term, when there is a large influx of 
students, and throughout the term, when the number of students needing 
placement is reduced. 

Although many agencies feel constrained by limited time and resources, 
they are concerned with maintaining the quality of their placement 
instrument. As a result, the challenge agencies face is to balance operational 
constraints with content considerations. Many agencies have developed 
their own placement instruments in an attempt to meet this challenge 13 ; 
however, it is clear from their responses that in order to appropriately 
place students into the levels specified in the Model Standards they need 
to address gaps in skill area coverage and the communicative nature of their 



12 See Butler, Weigle, Kahn, & Sato (forthcoming) for a discussion of the prototype development 
effort. 

13 Several of these agency-made tests were reviewed by project staff as input to the prototyping 
effort. (See Butler et al., forthcoming.) 



instruments. Assessing language proficiency in more than one skill area and 
focusing on the communicative use of language allow for more accurate 
placement decisions and better match the spirit of the Model Standards. By 
administering placement instruments that are aligned with the Model 
Standards, agencies send a clear message to both students and teachers about 
curricular goals and expectations. 

The detailed comments provided in response to the questionnaire 
confirmed that ESL placement issues are a major concern at a large number 
of agencies. Though agencies appreciated the opportunity to make their needs 
known, they are clearly looking for guidance from the state in bridging 
existing gaps in content coverage without exhausting their resources. To this 
end, this document, which describes current agency placement practices and 
concerns, serves as a first step in establishing guidelines for the development 
of placement instruments that are aligned with the Model Standards. 
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Appendix A 



Placement Procedures Questionnaire 



California Adult ESL Assessment Project 
Placement Procedures Questionnaire 



I. Demographic information 

Name of respondent 

Position Telephone ( ) . 

Name of agency 

Mailing address 

Number of adult ESL students enrolled 

Levels taught 

Number of adult ESL students needing placement per term 

To what extent are your current ESL courses aligned with the proficiency 
levels in the ESL Model Program Standards? 

Completely aligned Partially Not at all 

If not yet aligned, when do you anticipate that your courses will be aligned 
with the ESL Model Program Standards? 



II. Placement information 

1. What procedures do you use to place adult ESL students into your courses 
(Name of test and/or brief description of procedures) 



2. How was this procedure/test developed or selected? 



3. How long have you been using this procedure/test? 



4. How long does this procedure/test take to ad mini ster? 



5. How is the procedure/test scored (hand or machine)? 



6. How long does it take to score the procedure/test? 



7. How is placement done from the procedure/test (i.e., cutoff scores, profiles, 
etc.)? 



8. How often do you administer this procedure/test? 



9. What do you like most about your placement procedure/test? 



10. What problems, if any, do you have with your placement procedure/test? 



11. How do you feel about the accuracy of your current placement procedure? 



12. If students have been misplaced, how are adjustments made? How often 
does misplacement and subsequent adjustment occur? 



13. To what extent does your current procedure/test match the content of the 
ESL Model Program Standards in terms of both proficiency levels and 
communicative approach? 



14. Would you be willing to share your placement procedure/test with other 
agencies? 



15. Would you be willing to have your procedure/test evaluated by testing 
specialists? 



Thank you very much for completing this questionnaire. Please return it by 
December 3. 1993 in the enclosed envelope to: 

Dr. Frances A. Butler 

Center for the Study of Evaluation 

UCLA Graduate School of Education 

145 Moore Hall 

405 Hilgard Avenue 

Los Angeles, CA 90024-9969 



Appendix B 



List of Responding Agencies with ESL Programs 



Location 



Agency 



Location 



Agency 



ABC Adult School 


Cerritos 


Adelante Inc. 


Berkeley 


Adult School of Lucia Mar 
United School District 


Arroyo Grande 


Alameda Adult School 


Alameda 


Alameda County Library 
Adult Literacy 


Fremont 


Alhambra School District - 
Adult Division - Valley 
Adult Center 


San Gabriel 


Allan Hancock College 


Santa Maria 


Azusa Adult School 


Glendora 


Baldwin Park Adult and 
Continuing Education 
Program 


Baldwin Park 


Banning Adult School 


Banning 


Basic Adult Spanish 
Education 


Canoga Park 


Bassett Adult School 


La Puente 


Beaumont Adult School 


Beaumont 


Bellflower Adult School 


Bellflower 


Burbank Adult School 


Burbank 


Cabrillo Unified School 
District 


Half Moon Bay 


Capistrano Adult School 


Capistrano 

Valley 


Career Resources 
Development Center 


San Francisco 


Career Training and 
Education Center 


Yuba City 


Caregivers 


Oakland 


Castro Valley Adult School 


Castro Valley 


Center For Employment . 
Training 


Santa Maria 


Central Unified School 
District 


Fresno 


City College of San 
Francisco 


San Francisco 


Clovis Adult Education 


Clovis 


College of the Desert 


Palm Desert 



Compton Unified School 
District - Compton Adult 
School 


Compton 


Conejo Valley Adult School 


Thousand 

Oaks 


Corcoran Joint Unified 
School District - Corcoran 
Adult School 


Corcoran 


Corona-Norco Adult 
School 


Norco 


Culver City Adult School 


Culver City 


Dayle McIntosh Center for 
the Disabled 


Anaheim 


Del Norte Literacy 
Program 


Crescent City 


Delano Adult School 


Delano 


Dixon Unified School 
District 


Dixon 


Downey Adult School 


Downey 


Dublin School District 


Dublin 


East Side Adult Education 


San Jose 


Educational Options - 
Santa Clara Unified School 
District 


Santa Clara 


Edward Shands Adult 
School 


Oakland 


El Monte/Rosemead Adult 
School 


El Monte 


El Rancho Adult School 


Pico Rivera 


Elk Grove Unified School 
District - Adult Education 


Sacramento 


Escondido Adult School 


Escondido 


Esparto Unified School 
District - Esparto Adult 
School 


Esparto 


Eureka Adult School 


Eureka 


Fairfield/Suisun Adult 
School 


Fairfield 


Fremont Adult School 


Fremont 



Agency 



Location 



Fremont School for Adults - 
Sacramento City Unified 
School District 


Sacramento 


Fremont Union High 
School District - 
Community Education and 
Career Services 


Sunnyvale 


Fresno Adult School 


Fresno 


Fullerton Joint Union 
High School District - 
Alternative & Continuing 
Education 


Fullerton 


FV Literacy Center 


Midway City 


Garden Grove Adult 
Education 


Garden Grove 


Gilroy Adult Education 


Gilroy 


Glendale Community 
College 


Montrose 


Gonzales Adult School 


Soledad 


Grant Skills Center 


Sacramento 


Hacienda La Puente Adult 


Hacienda 


Education 


Heights 


Hayward Adult School 


Hayward 


Home Help for Hispanic 
Mothers 


Red Bluff 


HRA - Employment 
Services 


Willows 


Humboldt Literacy Project 


Eureka 


Huntington Beach Adult 


Huntington 


School 


Beach 


Inyo County Adult 
Education 


Independence 


Jefferson Union High 
School District - Adult 
Education Division 


Pacifica 


Jewish Vocational Service 


San Francisco 


Kerman Adult School 


Kerman 


Kern High School District - 
Bakersfield Adult School 


Bakersfield 


King City Adult School 


King City 



Kings Canyon Unified 
School District - Kings 
Canyon Adult School 


Reedley 


LAMP Literacy Program 


Monterey Park 


Laubach Literacy Council 
of San Diego 


San Diego 


Libreria Del Pueblo, Inc. 


San 

Bernardino 


Lindsay Adult School 


Lindsay 


Livermore Adult 
Education 


Livermore 


Long Beach Unified School 
District - Long Beach 
School for Adults 


Long Beach 


Los Angeles Unified 
School District 


Los Angeles 


Los Molinos Alternative 
Education Center 


Los Molinos 


Lynwood Adult School 


Lynwood 


Madera Adult School 


Madera 


Martinez Adult School 


Martinez 


Marysville Adult School 


Marysville 


Merced Adult School 


Merced 


Merced County Library - 
Literacy Center 


Merced 


Metropolitan Adult 
Education Project 


San Jose 


Milpitas Adult Education 


Milpitas 


Mira Costa Adult Learning 
Center 


Oceanside 


MODOC Read, Inc. 


Alturas 


Monrovia Adult School 


Monrovia 


Montebello Adult School 


Bell Gardens 


Montebello Literacy 
Program 


Montebello 


Monterey Adult School 


Monterey 


Moreno Valley 
Community Adult School 


Moreno Valley 


Morgan Hill Community 
Adult School 


Morgan Hill 



Agency 



Location 



Location Agency 



Mountain View - Los Altos 


Mountain 


Adult School 


View 


Mt. Diablo Adult Education 


Concord 


Mt. San Antonio College 


Walnut 


Napa Valley Adult School 


Napa 


Neighborhood Centers 
Adult School - Oakland 
Unified School District 


Oakland 


Nevada Union Adult 
Education 


Grass Valley 


New Haven Adult School 


Union City 


Newark Adult Education 


Newark 


Newport-Mesa Unified 
School District - Davis 
Adult Center 


Costa Mesa 


North Orange County 
Community College 
District 


Fullerton 


Norwalk/La Mirada Adult 
School 


Norwalk 


Oakland Evening Adult 
School 


Oakland 


Orange County Literacy 


Orange 


Oxnard Adult School 


Oxnard 


Pacific Grove Adult 
Education 


Pacific Grove 


Palo Alto Adult School 


Palo Alto 


Parlier Adult Education 
Program 


Parlier 


Pasadena City College - 
Community Skills Center 


Pasadena 


Perris Community Adult 


Perris 


Pleasanton Unified School 
District - Santa Rita Jail 


Pleasanton 


Pomona Unified School 
District - Adult and Career 
Education 


Pomona 


Poway Unified School 
District 


Poway 



Rancho Santiago 
Community College 
District - Continuing 
Education Division 


Santa Ana 


Redlands Adult School 


Redlands 


Riverside Adult School 


Riverside 


Robert T. Elliott 
Alternative Education 
Center 


Modesto 


Roseville Adult School 


Roseville 


Sacramento Chinese 
Community Service Center 


Sacramento 


Saddleback Valley Unified 
School District - Dept, of 
Adult Education 


Laguna Hills 


Salinas Adult School 


Salinas 


San Benito High Adult 
School 


Hollister 


San Bernardino City 


San 


Library Literacy Center 


Bernardino 


San Diego Community 
College District 


San Diego 


San Dieguito Adult School 


Encinitas 


San Gabriel Valley YWCA 


West Covina 


San Juan Unified School 
District 


Sacramento 


San Leandro Adult School 


San Leandro 


San Lorenzo Adult School 


San Lorenzo 


San Luis Obispo Literacy 


San Luis 


Council 


Obispo 


San Mateo Adult School 


San Mateo 


San Rafael City Schools 


San Rafael 


Sandigan California Inc. 


Sacramento 


Santa Barbara Community 
College - Continuing 
Education Division 


Santa Barbara 


Santa Cruz Adult 
Education 


Santa Cruz 


Santa Monica/Malibu 
Unified School District- 
Adult Education 


Santa Monica 



Agency 


Location 


Selma Adult School 


Selma 


Sequoia Adult School 


Menlo Park 


Shasta County Laubach 
Literacy Council 


Redding 


Simi Valley Adult School 


Simi Valley 


Siskiyou County Read 
Project, Inc. 


Yreka 


South Bay Literacy 
Council, Inc. 


Torrance 


Stockton School for Adults 


Stockton 


Sweetwater Unified High 
School District 


Chula Vista 


Tehama County Dept, of 
Education - Even Start 


Red Bluff 


Torrance Unified School 
District 


Torrance 


Tracy Adult School 


Tracy 


Tri-Community Adult 
Education 


West Covina 


Tulare Adult School 


Tulare 


Turlock Adult School 


Turlock 


Vacaville Unified School 
District 


Vacaville 


Ventura Adult - 
Continuing Education 


Ventura 


Victor Valley Community 
College 


Victorville 


Visalia Adult 


Visalia 


Vista Adult School 


Vista 


Watsonville/Aptos Adult 
School 


Watsonville 


West Contra Costa Unified 
School District - Richmond 
Adult 


Richmond 


Whittier Adult School 


Whittier 


Willows Learning Center 


Willows 


YMCA Literacy School 


San Francisco 



Appendix C 



List of Commercially Available Instruments 
Currently Being Used for ESL Placement 



List of Commercially Available Instruments 
Currently Being Used for ESL Placement 



I . Placement Tests Listed in the Model Standards 

Basic English Skills Test (BEST): Oral and Literacy 
Basic English Skills Test (BEST): Literacy 
(3) Basic English Skills Test (BEST): Oral 
(2) Combined English Language Skills Assessment (CELSA) 
Comprehensive English Language Tests (CELT) 

(2) Structure Tests - English Language (STEL) 

(2) Test of English Proficiency (TEPL) 

II. CASAS Tests 

(6) Adult Life Skills Appraisal 

(7) Agency-made (reading) 

(9) IRCA Pre-Enrollment Appraisal 
(11) Life Skills Survey Achievement 
(21) ESL Appraisal 
(26) Other CASAS tests 

III. Tests Associated with a Textbook Series or Teaching Method 
ESL Adult Literacy Scale 

English for a Changing World Placement Test 

Intercom 2000 Grammar Test 

New Horizons Placement Test 

Pathways to English Placement Tests 

Practical English Tests 

Real Life English Competency Test 



(2) Literacy Volunteers of America Placement 

(3) Side by Side Placement Test 

(4) Laubach Way to Reading 

IV. Other Commercial Tests 
Alemany Grammar Test 
CALPEP 

ESL Placement Test (Ilyin) 

ESL Oral Assessment (ESLOA) 

ESL Structure Test (ESLST) 

English Language Student Assessment (ELSA) 
Gates-MacGinite Reading Test (Level D) 
Henderson-Moriarity ESL/Literacy Placement 
Ilyin Proficiency Test 
Ilyin Structure Test 
John Test 

Steck-Vaughn Placement Survey 
Sullivan Reading Test 
V. McKinney Toehold 
(2) Quick Assessment 
(2) Idea Oral Language Proficiency Test 



( ) indicates the number of agencies that use the test when greater than 1. 
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